Learning in neural networks with local minima
نویسندگان
چکیده
منابع مشابه
Avoiding Local Minima in Feedforward Neural Networks by Simultaneous Learning
Feedforward neural networks are particularly useful in learning a training dataset without prior knowledge. However, weight adjusting with a gradient descent may result in the local minimum problem. Repeated training with random starting weights is among the popular methods to avoid this problem, but it requires extensive computational time. This paper proposes a simultaneous training method wi...
متن کاملLocal minima in training of neural networks
Training of a neural network is often formulated as a task of finding a “good” minimum of an error surface the graph of the loss expressed as a function of its weights. Due to the growing popularity of deep learning, the classical problem of studying the error surfaces of neural networks is now in the focus of many researchers. This stems from a long standing question. Given that deep networks ...
متن کاملLocal Minima and Plateaus in Multilayer Neural Networks
Local minima and plateaus pose a serious problem in learning of neural networks. We investigate the geometric structure of the parameter space of three-layer perceptrons in order to show the existence of local minima and plateaus. It is proved that a critical point of the model with H ? 1 hidden units always gives a critical point of the model with H hidden units. Based on this result , we prov...
متن کاملExponentially Vanishing Sub-optimal Local Minima in Multilayer Neural Networks
Background: Statistical mechanics results (Dauphin et al. (2014); Choromanska et al. (2015)) suggest that local minima with high error are exponentially rare in high dimensions. However, to prove low error guarantees for Multilayer Neural Networks (MNNs), previous works so far required either a heavily modified MNN model or training method, strong assumptions on the labels (e.g., “near” linear ...
متن کاملOn the problem of local minima in recurrent neural networks
Many researchers have recently focused their efforts on devising efficient algorithms, mainly based on optimization schemes, for learning the weights of recurrent neural networks. As in the case of feedforward networks, however, these learning algorithms may get stuck in local minima during gradient descent, thus discovering sub-optimal solutions. This paper analyses the problem of optimal lear...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Physical Review A
سال: 1992
ISSN: 1050-2947,1094-1622
DOI: 10.1103/physreva.46.5221